<!DOCTYPE html>
<html>

<head>
	<meta charset="utf-8">
	<meta name="viewport" content="width=device-width, initial-scale=1, maximum-scale=1">
	<meta name="theme-color" content="#33474d">
	<title>查看mysql主从复制延迟和数据中断 | 失落的乐章</title>
	<link rel="stylesheet" href="/css/style.css" />
	
      <link rel="alternate" href="/atom.xml" title="失落的乐章" type="application/atom+xml">
    
</head>

<body>

	<header class="header">
		<nav class="header__nav">
			
				<a href="/archives" class="header__link">Archive</a>
			
				<a href="/tags" class="header__link">Tags</a>
			
				<a href="/atom.xml" class="header__link">RSS</a>
			
		</nav>
		<h1 class="header__title"><a href="/">失落的乐章</a></h1>
		<h2 class="header__subtitle">技术面前，永远都是学生。</h2>
	</header>

	<main>
		<article>
	
		<h1>查看mysql主从复制延迟和数据中断</h1>
	
	<div class="article__infos">
		<span class="article__date">2017-10-12</span><br />
		
		
			<span class="article__tags">
			  	<a class="article__tag-link" href="/tags/MySQL/">MySQL</a>
			</span>
		
	</div>

	

	
		<p>&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;查看mySQL延迟的方法，查看了多个案例，大家众说纷纭，意见差不多一致。如下也是我参考别人经验做的一些测试，希望能检测到mysql复制延迟、数据中断。</p>
<h2 id="方法一：查看Seconds-Behind-Master"><a href="#方法一：查看Seconds-Behind-Master" class="headerlink" title="方法一：查看Seconds_Behind_Master"></a>方法一：查看Seconds_Behind_Master</h2><p>&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;该参数有如下值：</p>
<ul>
<li>NULL  表示io_thread或sql_thread有一个发生故障，就是说该线程的Running状态时No，而非Yes</li>
<li>0     表示主从复制良好，没有lag存在</li>
<li>正值  表示主从已出现延时，数字越大表示从库落后主库越多</li>
<li>负值  很罕见，是一个BUG，按理说不应该出现</li>
</ul>
<p>&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;该方法是使用命令show slave status,通过比较SQL THREAD接受events时间的时间戳与IO THREAD执行事件events时间戳的差值–秒数，来确定slave落后于master多少，如主从时间不同，改时间的计算不受影响。</p>
<p>&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;众所周知备库relay-log和主库的bin-log里的内容一样，真正和主库有关两的是io_thread，当主库I/O负载很大或网络阻塞时，io_thread不能及时复制binlog，而sql_thread一直能跟上io_thread的脚步，这时seconds_behind_master的值是0，实际上却不是，这时用该值作为延迟参考则不准。</p>
<figure class="highlight bash"><table><tr><td class="gutter"><pre><div class="line">1</div></pre></td><td class="code"><pre><div class="line">change master to master_host=<span class="string">'192.168.2.7'</span>,master_user=<span class="string">'tongbu'</span>,master_password=<span class="string">'123456'</span>,master_log_file=<span class="string">'mysql-bin.000008'</span>,master_log_pos=291263843;</div></pre></td></tr></table></figure>
<h2 id="方法二：使用pt-heartbeat工具"><a href="#方法二：使用pt-heartbeat工具" class="headerlink" title="方法二：使用pt-heartbeat工具"></a>方法二：使用pt-heartbeat工具</h2><p>&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;该工具可以计算出MySQL复制或者是PostgreSQL,它可以更新master或者监控复制。它还可以从f 读取配置。它借助timestmp的比较实现的，首先需要保证主从服务器时间必须要保持一致，通过与相同的一个NTP server同步时钟。它需要在主库上创建一个heartbeat的表，里面的时间戳ts就是当前的时间戳 now()，该结构也会被复制到从库上。表建好以后，会在主库上以后台进程的模式去执行一行更新操作的命令，定期去向表中的插入数据，这 个周期默认为1 秒，同时从库也会在后台执行一个监控命令，与主库保持一致的周期+0.5S（默认0.5S延迟检查）去比较，复制过来记录的ts值与主库上的同一条ts值，差值为0表示无延时，差值越大表示 延时的秒数越多。</p>
<h2 id="使用工具前提："><a href="#使用工具前提：" class="headerlink" title="使用工具前提："></a>使用工具前提：</h2><ol>
<li>在主库上建立heartbeat表</li>
</ol>
<figure class="highlight bash"><table><tr><td class="gutter"><pre><div class="line">1</div></pre></td><td class="code"><pre><div class="line">pt-heartbeat -h localhost -D <span class="built_in">test</span>  --create-table --update</div></pre></td></tr></table></figure>
<ol>
<li>更新主库上的heartbeat</li>
</ol>
<figure class="highlight bash"><table><tr><td class="gutter"><pre><div class="line">1</div></pre></td><td class="code"><pre><div class="line">pt-heartbeat -D <span class="built_in">test</span> --master-server-id=1 --update</div></pre></td></tr></table></figure>
<ol>
<li>在从库上监控复制延迟</li>
</ol>
<figure class="highlight bash"><table><tr><td class="gutter"><pre><div class="line">1</div><div class="line">2</div><div class="line">3</div><div class="line">4</div><div class="line">5</div><div class="line">6</div><div class="line">7</div><div class="line">8</div><div class="line">9</div><div class="line">10</div><div class="line">11</div><div class="line">12</div><div class="line">13</div><div class="line">14</div><div class="line">15</div><div class="line">16</div><div class="line">17</div></pre></td><td class="code"><pre><div class="line">pt-heartbeat --user=tongbu --password=<span class="string">'123456'</span> -D <span class="built_in">test</span> --monitor -h 192.168.2.9 --<span class="built_in">print</span>-master-server-id</div><div class="line">0.00s [  0.00s,  0.00s,  0.00s ] 1</div><div class="line">0.00s [  0.00s,  0.00s,  0.00s ] 1</div><div class="line">0.00s [  0.00s,  0.00s,  0.00s ] 1</div><div class="line">0.00s [  0.00s,  0.00s,  0.00s ] 1</div><div class="line">0.00s [  0.00s,  0.00s,  0.00s ] 1</div><div class="line">0.00s [  0.00s,  0.00s,  0.00s ] 1</div><div class="line">0.00s [  0.00s,  0.00s,  0.00s ] 1</div><div class="line">0.00s [  0.00s,  0.00s,  0.00s ] 1</div><div class="line">0.00s [  0.00s,  0.00s,  0.00s ] 1</div><div class="line">0.01s [  0.00s,  0.00s,  0.00s ] 1</div><div class="line">0.00s [  0.00s,  0.00s,  0.00s ] 1</div><div class="line">0.00s [  0.00s,  0.00s,  0.00s ] 1</div><div class="line">0.00s [  0.00s,  0.00s,  0.00s ] 1</div><div class="line">0.00s [  0.00s,  0.00s,  0.00s ] 1</div><div class="line">0.00s [  0.00s,  0.00s,  0.00s ] 1</div><div class="line">0.00s [  0.00s,  0.00s,  0.00s ] 1</div></pre></td></tr></table></figure>
<p>&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;当然还有其他一些操作命令：</p>
<p>&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;将主库上的update使用守护进程方式调度</p>
<figure class="highlight bash"><table><tr><td class="gutter"><pre><div class="line">1</div></pre></td><td class="code"><pre><div class="line">pt-heartbeat -D <span class="built_in">test</span> --master-server-id=1 --update --daemonize</div></pre></td></tr></table></figure>
<p>&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;修改主库上的更新时间间隔为2s</p>
<figure class="highlight bash"><table><tr><td class="gutter"><pre><div class="line">1</div></pre></td><td class="code"><pre><div class="line">pt-heartbeat -D <span class="built_in">test</span> --update --daemonize --interval=2</div></pre></td></tr></table></figure>
<p>&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;修改主库上的pt-heartbeat守护进程</p>
<figure class="highlight bash"><table><tr><td class="gutter"><pre><div class="line">1</div><div class="line">2</div><div class="line">3</div><div class="line">4</div></pre></td><td class="code"><pre><div class="line">pt-heartbeat --stop</div><div class="line">Successfully created file /tmp/pt-heartbeat-sentinel</div><div class="line"></div><div class="line">rm -rf /tmp/pt-heartbeat-sentinel</div></pre></td></tr></table></figure>
<p>&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;单词查看从库上的延迟情况</p>
<figure class="highlight bash"><table><tr><td class="gutter"><pre><div class="line">1</div><div class="line">2</div></pre></td><td class="code"><pre><div class="line">pt-heartbeat --user=tongbu --password=<span class="string">'123456'</span> -D <span class="built_in">test</span>  -h 192.168.2.9 --check</div><div class="line">0.00</div></pre></td></tr></table></figure>
<p>&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;使用守护进程监控从库并输出日志</p>
<figure class="highlight bash"><table><tr><td class="gutter"><pre><div class="line">1</div></pre></td><td class="code"><pre><div class="line">pt-heartbeat --user=tongbu --password=<span class="string">'123456'</span> -h 192.168.2.9 -D <span class="built_in">test</span> --master-server-id=1 --monitor --<span class="built_in">print</span>-master-server-id --daemonize --<span class="built_in">log</span>=/var/<span class="built_in">log</span>/pt_slave_lag.log</div></pre></td></tr></table></figure>
<p>&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;如下是脚本的方式，只不过使用脚本的方式实现的。</p>
<figure class="highlight bash"><table><tr><td class="gutter"><pre><div class="line">1</div><div class="line">2</div><div class="line">3</div><div class="line">4</div><div class="line">5</div><div class="line">6</div><div class="line">7</div><div class="line">8</div><div class="line">9</div><div class="line">10</div><div class="line">11</div><div class="line">12</div><div class="line">13</div><div class="line">14</div><div class="line">15</div><div class="line">16</div><div class="line">17</div><div class="line">18</div><div class="line">19</div><div class="line">20</div><div class="line">21</div><div class="line">22</div><div class="line">23</div><div class="line">24</div><div class="line">25</div><div class="line">26</div><div class="line">27</div><div class="line">28</div><div class="line">29</div><div class="line">30</div><div class="line">31</div><div class="line">32</div><div class="line">33</div><div class="line">34</div><div class="line">35</div><div class="line">36</div><div class="line">37</div><div class="line">38</div><div class="line">39</div><div class="line">40</div><div class="line">41</div><div class="line">42</div><div class="line">43</div><div class="line">44</div><div class="line">45</div><div class="line">46</div><div class="line">47</div><div class="line">48</div><div class="line">49</div><div class="line">50</div><div class="line">51</div><div class="line">52</div><div class="line">53</div><div class="line">54</div><div class="line">55</div><div class="line">56</div><div class="line">57</div><div class="line">58</div><div class="line">59</div><div class="line">60</div><div class="line">61</div><div class="line">62</div><div class="line">63</div><div class="line">64</div><div class="line">65</div><div class="line">66</div><div class="line">67</div><div class="line">68</div><div class="line">69</div><div class="line">70</div><div class="line">71</div><div class="line">72</div><div class="line">73</div><div class="line">74</div><div class="line">75</div><div class="line">76</div><div class="line">77</div><div class="line">78</div><div class="line">79</div><div class="line">80</div><div class="line">81</div><div class="line">82</div><div class="line">83</div><div class="line">84</div><div class="line">85</div><div class="line">86</div><div class="line">87</div><div class="line">88</div><div class="line">89</div><div class="line">90</div><div class="line">91</div><div class="line">92</div><div class="line">93</div><div class="line">94</div><div class="line">95</div><div class="line">96</div><div class="line">97</div><div class="line">98</div><div class="line">99</div><div class="line">100</div><div class="line">101</div><div class="line">102</div><div class="line">103</div><div class="line">104</div><div class="line">105</div><div class="line">106</div><div class="line">107</div><div class="line">108</div><div class="line">109</div><div class="line">110</div><div class="line">111</div></pre></td><td class="code"><pre><div class="line"><span class="meta">#!/bin/sh</span></div><div class="line"><span class="comment">#description:check slave replication delay</span></div><div class="line">PT=`<span class="built_in">which</span> pt-heartbeat`</div><div class="line">ADMIN=`<span class="built_in">which</span> mysqladmin`</div><div class="line">MYSQL=`<span class="built_in">which</span> mysql`</div><div class="line">WARN=10</div><div class="line">CRITIC=20</div><div class="line">COMMON=3</div><div class="line">MASTERID=1</div><div class="line"><span class="comment">###############</span></div><div class="line"><span class="function"><span class="title">watch_update</span></span>()&#123;</div><div class="line">RE=`ps -ef|grep <span class="variable">$PT</span>|grep -v grep|grep <span class="string">"\--update"</span>`</div><div class="line"><span class="keyword">if</span> [ ! -n <span class="string">"<span class="variable">$RE</span>"</span> ];<span class="keyword">then</span></div><div class="line">   <span class="variable">$PT</span> -D <span class="built_in">test</span> --master-server-id=<span class="variable">$MASTERID</span> --update --daemonize</div><div class="line"><span class="keyword">fi</span></div><div class="line">&#125;</div><div class="line"><span class="comment">################</span></div><div class="line"><span class="function"><span class="title">watch_mysql</span></span>()&#123;</div><div class="line">SAFE_STATUS=`ps -ef|grep -w mysqld_safe|grep -v grep`</div><div class="line">MYSQLD_STATUS=`ps -ef|grep -w mysqld|grep -v grep`</div><div class="line"><span class="keyword">if</span> [ ! -n <span class="string">"<span class="variable">$SAFE_STATUS</span>"</span> ];<span class="keyword">then</span></div><div class="line">   <span class="built_in">echo</span> <span class="string">"Mysqld_safe doesn't running,Please check your mysqld_safe status"</span></div><div class="line">   <span class="built_in">exit</span> 1</div><div class="line"><span class="keyword">fi</span></div><div class="line"><span class="keyword">if</span> [ ! -n <span class="string">"<span class="variable">$MYSQLD_STATUS</span>"</span> ];<span class="keyword">then</span></div><div class="line">   <span class="built_in">echo</span> <span class="string">"Mysqld program status --stop,Please check your mysqld status "</span></div><div class="line">   <span class="built_in">exit</span> 1</div><div class="line"><span class="keyword">fi</span></div><div class="line">&#125;</div><div class="line"><span class="comment">####################</span></div><div class="line"><span class="function"><span class="title">watch_mysql_slave</span></span>()&#123;</div><div class="line">REP_STATUS=`<span class="variable">$ADMIN</span> processlist|grep <span class="string">"Binlog Dump"</span>`</div><div class="line"><span class="keyword">if</span> [ ! -n <span class="string">"<span class="variable">$REP_STATUS</span>"</span> ];<span class="keyword">then</span></div><div class="line">    <span class="built_in">echo</span> <span class="string">"slave process doesn't running,Please check your replication"</span></div><div class="line">    <span class="built_in">exit</span> 1</div><div class="line"><span class="keyword">fi</span></div><div class="line">&#125;</div><div class="line"><span class="comment">#################</span></div><div class="line"><span class="function"><span class="title">enter_slave_info</span></span>()&#123;</div><div class="line">  <span class="built_in">echo</span> <span class="string">"please enter your slave username:"</span></div><div class="line">  <span class="built_in">read</span> NAME</div><div class="line">  <span class="keyword">if</span> [  -n <span class="string">"<span class="variable">$NAME</span>"</span> ];<span class="keyword">then</span></div><div class="line">      <span class="built_in">echo</span> <span class="string">" you enter username is: <span class="variable">$NAME</span>"</span></div><div class="line">  <span class="keyword">fi</span></div><div class="line">  </div><div class="line">  <span class="built_in">echo</span> <span class="string">"please enter your slave user password:"</span></div><div class="line">  <span class="built_in">read</span> PASS</div><div class="line">  <span class="keyword">if</span> [ -n <span class="string">"<span class="variable">$PASS</span>"</span> ];<span class="keyword">then</span></div><div class="line">      <span class="built_in">echo</span> <span class="string">"you enter user password is: <span class="variable">$PASS</span>"</span></div><div class="line">  <span class="keyword">fi</span>  </div><div class="line">  </div><div class="line">  <span class="built_in">echo</span> <span class="string">"please enter your slave hostname or address:"</span></div><div class="line">  <span class="built_in">read</span> ADDR</div><div class="line">  <span class="keyword">if</span> [ -n <span class="string">"<span class="variable">$ADDR</span>"</span> ];<span class="keyword">then</span></div><div class="line">      <span class="built_in">echo</span> <span class="string">"you enter slave hostname or address is: <span class="variable">$ADDR</span>"</span></div><div class="line">  <span class="keyword">fi</span></div><div class="line">&#125;</div><div class="line"><span class="comment">##################</span></div><div class="line"><span class="function"><span class="title">watch_slave_delay</span></span>()&#123;</div><div class="line"><span class="comment">#  echo "please enter your watch options(1.check 2.monitor)"</span></div><div class="line"><span class="comment">#  read OPTION</span></div><div class="line"><span class="comment">#  if  [ $OPTION -eq 1 ];then</span></div><div class="line">      SLAVE_DELAY=`<span class="variable">$PT</span> --user=<span class="variable">$NAME</span> --password=<span class="string">"<span class="variable">$PASS</span>"</span> -h <span class="variable">$ADDR</span> -D <span class="built_in">test</span> --master-server-id=<span class="variable">$MASTERID</span> --check --<span class="built_in">print</span>-master-server-id`</div><div class="line"><span class="comment">#  elif [ $OPTION -eq 2 ];then</span></div><div class="line"><span class="comment">#      `$PT --user=$NAME --password="PASS" -h $ADDR -D test --master-server-id=$MASTERID --monitor --print-master-server-id`</span></div><div class="line"><span class="comment">#  else</span></div><div class="line"><span class="comment">#       echo "your enter are error,now EXIT"</span></div><div class="line"><span class="comment">#       exit 1</span></div><div class="line"><span class="comment">#  fi</span></div><div class="line"><span class="keyword">if</span> [ ! -n <span class="string">"<span class="variable">$SLAVE_DELAY</span>"</span> ];<span class="keyword">then</span></div><div class="line">   <span class="built_in">echo</span> <span class="string">"Your pt-heartbeat tool must haven't install or you username,password,slave hostname ERROR"</span></div><div class="line">   <span class="built_in">exit</span> 1</div><div class="line"><span class="keyword">else</span></div><div class="line">   <span class="built_in">echo</span> <span class="string">"DELAY TIME:  <span class="variable">$SLAVE_DELAY</span>"</span></div><div class="line"><span class="keyword">fi</span>  </div><div class="line">&#125;</div><div class="line"></div><div class="line"><span class="comment">##################################</span></div><div class="line"><span class="function"><span class="title">watch_slave_interrupt</span></span>()&#123;</div><div class="line">  INT_RE=`<span class="variable">$MYSQL</span> -s -u <span class="variable">$NAME</span> -p<span class="string">"<span class="variable">$PASS</span>"</span> -h <span class="variable">$ADDR</span> -e <span class="string">'show slave status\G'</span>|grep <span class="string">"Last_Error:"</span>`</div><div class="line">  <span class="keyword">if</span> [ -n <span class="string">"<span class="variable">$INT_RE</span>"</span> ];<span class="keyword">then</span></div><div class="line">    <span class="built_in">echo</span> <span class="string">"<span class="variable">$INT_RE</span>"</span></div><div class="line">  <span class="keyword">else</span></div><div class="line">    <span class="built_in">echo</span>  <span class="string">"INTERUUPT Info: "</span></div><div class="line">    <span class="built_in">echo</span>  <span class="string">"<span class="variable">$INT_RE</span>"</span></div><div class="line">  <span class="keyword">fi</span></div><div class="line">&#125;</div><div class="line"><span class="comment">########################</span></div><div class="line"><span class="function"><span class="title">WATCH</span></span>()&#123;</div><div class="line">watch_slave_delay</div><div class="line">watch_slave_interrupt</div><div class="line">&#125;</div><div class="line"></div><div class="line"><span class="comment">#########################</span></div><div class="line"><span class="function"><span class="title">START_WATCH</span></span>()&#123;</div><div class="line">  watch_mysql</div><div class="line">  watch_mysql_slave</div><div class="line">  watch_update</div><div class="line">  enter_slave_info </div><div class="line">&#125;</div><div class="line"><span class="comment">##########################</span></div><div class="line"><span class="function"><span class="title">LOOP_WATCH</span></span>()&#123;</div><div class="line">  START_WATCH</div><div class="line"><span class="built_in">echo</span> <span class="string">"+++++++++++++ DELAY INFO  ++++++++++++++++"</span></div><div class="line">  <span class="keyword">while</span> <span class="literal">true</span>;<span class="keyword">do</span></div><div class="line">    WATCH</div><div class="line">    sleep 30</div><div class="line">  <span class="keyword">done</span></div><div class="line">&#125;</div><div class="line"><span class="comment">##########################</span></div><div class="line">LOOP_WATCH</div></pre></td></tr></table></figure>
<p>&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;脚本运行结果如下：</p>
<figure class="highlight bash"><table><tr><td class="gutter"><pre><div class="line">1</div><div class="line">2</div><div class="line">3</div><div class="line">4</div><div class="line">5</div><div class="line">6</div><div class="line">7</div><div class="line">8</div><div class="line">9</div><div class="line">10</div><div class="line">11</div><div class="line">12</div><div class="line">13</div><div class="line">14</div><div class="line">15</div><div class="line">16</div><div class="line">17</div><div class="line">18</div><div class="line">19</div><div class="line">20</div><div class="line">21</div><div class="line">22</div><div class="line">23</div></pre></td><td class="code"><pre><div class="line">[root@mjx mjx]<span class="comment"># ./delay.sh </span></div><div class="line">please enter your slave username:</div><div class="line">tongbu</div><div class="line">you enter username is: tongbu</div><div class="line">please enter your slave user password:</div><div class="line">123456</div><div class="line">you enter user password is: 123456</div><div class="line">please enter your slave hostname or address:</div><div class="line">192.168.2.9</div><div class="line">you enter slave hostname or address is: 192.168.2.9</div><div class="line">+++++++++++++ DELAY INFO  ++++++++++++++++</div><div class="line">DELAY TIME:  0.36 1</div><div class="line">Warning: Using a password on the <span class="built_in">command</span> line interface can be insecure.</div><div class="line">                   Last_Error: </div><div class="line">DELAY TIME:  0.96 1</div><div class="line">Warning: Using a password on the <span class="built_in">command</span> line interface can be insecure.</div><div class="line">                   Last_Error: </div><div class="line">DELAY TIME:  0.12 1</div><div class="line">Warning: Using a password on the <span class="built_in">command</span> line interface can be insecure.</div><div class="line">                   Last_Error: </div><div class="line">DELAY TIME:  1.91 1</div><div class="line">Warning: Using a password on the <span class="built_in">command</span> line interface can be insecure.</div><div class="line">                   Last_Error:</div></pre></td></tr></table></figure>
<p>&#160;&#160;&#160;&#160;&#160;&#160;&#160;&#160;但是输出结果没法把报错屏蔽掉。</p>

	

	
		<span class="different-posts"><a href="/2017/10/12/MySQL/34. 查看mysql主从复制延迟和数据中断/" onclick="window.history.go(-1); return false;">⬅️ Go back </a></span>

	

</article>

	</main>

	<footer class="footer">
	<div class="footer-content">
		
	      <div class="footer__element">
	<p>Hi there, <br />welcome to my Blog glad you found it. Have a look around, will you?</p>
</div>

	    
	      <div class="footer__element">
	<h5>Check out</h5>
	<ul class="footer-links">
		<li class="footer-links__link"><a href="/archives">Archive</a></li>
		
		  <li class="footer-links__link"><a href="/atom.xml">RSS</a></li>
	    
		<li class="footer-links__link"><a href="/about">about page</a></li>
		<li class="footer-links__link"><a href="/tags">Tags</a></li>
		<li class="footer-links__link"><a href="/categories">Categories</a></li>
	</ul>
</div>

	    

		<div class="footer-credit">
			<span>© 2017 失落的乐章 | Powered by <a href="https://hexo.io/">Hexo</a> | Theme <a href="https://github.com/HoverBaum/meilidu-hexo">MeiliDu</a></span>
		</div>

	</div>


</footer>



</body>

</html>
